The present disclosure generally relates to load testing IT products. The disclosed embodiments relate more specifically to a system, apparatus, method, and computer program product for dynamically loading IT products and scaling those loads in a predictive manner.
In the information technology (IT) industry, the term “load testing” generally refers to the practice of modeling the expected usage of an IT product (e.g., an application, a website, a virtual machine, a server, etc.) by simulating multiple consumers accessing that IT product concurrently. Such load testing typically is utilized to measure an IT product's quality of service (QoS) performance with simulated consumer behavior prior to releasing the IT product for consumption by actual consumers. Such consumers may included, for example, a real person or another computing device. And by load testing the IT product in that way, the IT product provider may ensure that the IT product will function properly when it is released for consumption by those consumers.
Consumer behavior may be simulated by having live and/or virtual users access the IT product and mimic the behavior of real person consumers. For example, the IT product provider may procure a large number of computing devices that are configured to access the IT product (e.g., desktop computers, laptop computers, tablet computers, smart phones, etc.) and recruit a corresponding number of live users to begin accessing the IT product with those computing devices. Those live users then would engage in various interactions with the IT product that mimic the types of interactions expected to be made with the IT product by real persons consuming the services of the IT product.
In the foregoing example, the number of computing devices that must be procured generally corresponds to the number of consumers for which the IT product provider would like to simulate the subject consumer behavior. Thus, to simulate the behavior of a thousand (1,000) consumers, the IT product provider would be required to procure a thousand (1,000) computing devices that are configured to access the IT product. Moreover, the IT product provider also would be required to recruit a corresponding number of users to operate those computing devices to simulate the subject consumer behavior. Such a task presents a significant logistical challenge, which grows more and more difficult to address as the number of consumers for which the IT product provider would like to simulate the subject
The foregoing logistical challenge may be partially addressed by provisioning a plurality computing devices load generating software that executes various scripts that mimic the types of interactions expected to be made with the IT product by real persons and/or other computing devices (e.g., LISA brand load generating software from CA Technologies. LOADRUNNER brand load generating software from Hewlett-Packard, WEBLOAD brand load generating software from RadView Software, etc.). Such load generating software allows a plurality of virtual users to be invoked at each load generating computing device, rather than providing a single live user at each such computing device, thereby reducing the number of computing devices required to conduct a load test. It also may allow a plurality of other computing devices to be emulated at each load generating computing device. Nevertheless, it remains difficult for test engineers to predict the number of virtual users that can be invoked at each load generating computing device because that number varies based on the nature of the test, the hardware employed on the computing device, and/or the type of other computing device being emulated. It is similarly difficult for test engineers to predict the number of instances of virtual users that will be required to satisfactorily test the subject IT product. Accordingly, test engineers often over-allocate or under-allocate resources for a particular load test, which results in the inefficient duplication of time and effort to set up and re-run a new load test when the initial load test fails.
For example, a test engineer may believe that a particular IT product will reach a predefined performance goal (e.g., CPU usage, memory usage, disk I/O, response time, throughput, etc.) when concurrently accessed by thirty thousand (30,000) real person consumers. The test engineer also may believe that each computing device may simulate one thousand (1,000) of those real person consumers. Accordingly, the test engineer would provision thirty (30) computing devices with load generating software for use in running the load test.
Such provisioning may involve installing the load generating software on the computing devices of various employees of the IT product provider so that the IT product provider is not required to procure new computing devices just to run the load test. Nevertheless, to use the computing devices of those employees, the test engineer not only must install the load generating software when those computing devices are not being used by those employees (e.g., when the employees are on break or otherwise not working), he/she also must run the load test for longer periods when those computing devices are not being used (e.g., after regular work hours). And when the test engineer incorrectly guesses the number consumers required to reach the predefined performance goal of the IT product and/or the number of consumers that may be simulated with each computing device, the predefined performance goal of the IT product may not be reached. The test engineer then would be required to repeat the preceding steps at least one more time until the load test is scaled properly, thereby duplicating timely work and consuming more resources.